Add session::image::thumbhash image placeholders - #172
Merged
Merged
Conversation
mpretty-cyro
approved these changes
Sep 21, 2026
It pins clang-format-19; the system clang-format is a different version and produces different output, which fails the CI format check.
ThumbHash (https://github.com/evanw/thumbhash, MIT) encodes a ~20-25 byte placeholder that a receiving client renders as a blurred preview while the real attachment downloads. It is a better fit than BlurHash on every axis we care about: natively binary rather than base83, alpha support, and it spends its bits like a codec (7x7 DCT on luminance, 3x3 per chroma axis, 5x5 on alpha) instead of splitting them evenly across R, G and B. Measured against 13 photos, scored by RMSE against the original downsampled to 32px: thumbhash 20.5 bytes / 29.25, raw 4x4 pixels 48 bytes / 30.87, BlurHash 4x3 28 bytes / 36.68. Smallest and best of everything tried. There is no upstream C or C++ implementation, so this is a port, with three deliberate departures: - The DCT basis goes through src/image/det_trig.hpp rather than std::cos, and every a*b+c is an explicit std::fma. Neither std::cos's accuracy nor whether the compiler contracts a multiply-add is fixed by the standard, and both change the emitted hash: V8's cos differs from glibc's by 1 ulp on ~3.5% of the arguments this DCT uses, which alters ~7% of hashes. Since a thumbhash is sent to other people, that would make it a weak fingerprint of the sending platform. Exact integer argument reduction also happens to be 27x more accurate than the reference formulation, which hands libm a thrice-rounded angle. Verified identical across 7 GCC and 2 Clang configurations spanning -O0..-O3, -march=native, -mfma and -ffp-contract both fast and off. - The basis tables are hoisted out of the innermost loops; the reference rebuilds them per (cx, cy) when encoding and per pixel when decoding. Accumulation order is untouched, so this is bit-identical, and about 3x faster each way: encode 0.11ms at 32x32, decode 0.085ms at 32x24. - Decoding is resolution-independent, so there is no upscale-and-blur step and no image library needed on the receive path. Keep the decode small and let the UI scale it: at 32px, upscaling 10x with any linear filter differs from a full-size decode by RMSE 0.87, which is imperceptible. The hash therefore differs from upstream encoders by at most 1 in an individual 4-bit AC coefficient. Decoders remain fully interoperable; nothing requires two encoders to agree. THUMBHASH_REFERENCE_COS switches the basis back to the upstream std::cos formulation, which is what the algorithm is defined as and what its published vectors were produced with; it is there to keep that legible and measurable, and must not be defined in production. Two tests guard the parts that would otherwise fail silently. The pinned hash vectors catch a build that reintroduces a platform dependency, rather than letting it leak. The det_trig case checks the hand-written Taylor coefficients against libm across the DCT's whole argument set, because a mistranscribed factorial still yields smooth, plausible output -- just with the wrong values in every hash. The API is shaped so the safe path is the obvious one. decode() takes explicit dimensions, because a hash carries no usable record of the source's shape: what it stores is the DCT component counts, which a decoder needs in order to know how many luminance AC coefficients precede the chroma terms. That parameter tracks the shape loosely, so upstream exposes it as an aspect ratio, but one side is always pinned at the per-channel maximum and the complete set of results is 7/n for n in 1..7 and reciprocals (13 values), or 5/n for n in 1..5 with alpha. It also saturates, so a 1000x100 banner reads as 7.0 rather than 10.0 and a 1x100 sliver is wrong by a factor of fourteen. It is therefore named component_aspect_ratio(), and the overload that guesses an output shape from it is decode_unsized(); both are documented as diagnostics for a hash that arrives with no metadata, not as part of a display path. valid() and expected_size() are for a carrier storing peer-supplied values. A hash's length is not merely bounded but fully determined by its own header -- only six lengths are reachable at all -- so a receive path can check exact structural validity for the cost of a few bit extractions, which is strictly stronger than a length cap that any blob of the right size would pass. Note for the libvips work: encode() takes RGBA at <=100x100, so the caller does the downscaling, and the scaler is now the weakest link for reproducibility -- two clients that downscale the same photo differently produce different hashes.
Bilb
reviewed
Sep 21, 2026
Review fixes for session-foundation#172. decode() indexed its output with an `int` that wrapped at 2^29 output pixels, after which the stores went outside the buffer: decode(hash, 32768, 16384) is a signed overflow under UBSan, and decode(hash, 32768, 20480) segfaults. The index is now size_t, the byte count is computed in uint64_t and rejected if it would not fit a size_t (which is what would otherwise under-allocate on a 32-bit target and overflow the same way at a much lower pixel count), and the dimensions are bounded to int32 so the `int` loop counters and det_trig's 2*i+1 cannot overflow either. No policy cap: a big decode is merely slow, as intended -- the header now says to decode at 100x100 or less and scale the result up with libvips, which suits a blurred image well. valid() accepted a stored component count of 0. No conforming encoder emits one -- both this encoder and upstream clamp to max(1, ...) -- but it is a bit-field a peer controls, and while the decoder's max(3, ...) clamp waves it through, component_aspect_ratio reads the field unclamped and rejects it. A carrier that stored a value on the strength of valid() could then get an exception out of the documented no-metadata path. expected_size() now rejects it, so the trust boundary agrees with the rest of the API. Nothing tested the decoder. The pinned vectors cover the encoder; the round-trip test checks buffer sizes but no pixels, resolution-independence compares the decoder against itself, and the average-colour test compares two quantities that both derive from the same DC terms. A decoder producing consistently wrong pixels passed the entire suite. Its output is now pinned at 8x6 for the same eight vectors. Also: extract lpqa_to_rgba8, which decode() and average_rgba() had character-for-character in common, so the flat-colour placeholder cannot drift from the image it stands in for; fix a comment naming decode_at, which was renamed to decode; and correct a test comment saying "eight lengths" about a set of six. The encoder is untouched: its pinned vectors are unchanged across -O0..-O3, -march=native, -mfma and -ffp-contract fast/off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ThumbHash (https://github.com/evanw/thumbhash, MIT) encodes a ~20-25 byte placeholder that a receiving client renders as a blurred preview while the real attachment downloads. It is a better fit than BlurHash on every axis we care about: natively binary rather than base83, alpha support, and it spends its bits like a codec (7x7 DCT on luminance, 3x3 per chroma axis, 5x5 on alpha) instead of splitting them evenly across R, G and B.
Measured against 13 photos, scored by RMSE against the original downsampled to 32px: thumbhash 20.5 bytes / 29.25, raw 4x4 pixels 48 bytes / 30.87, BlurHash 4x3 28 bytes / 36.68. Smallest and best of everything tried.
There is no upstream C or C++ implementation, so this is a port, with three deliberate departures:
The DCT basis goes through src/image/det_trig.hpp rather than std::cos, and every a*b+c is an explicit std::fma. Neither std::cos's accuracy nor whether the compiler contracts a multiply-add is fixed by the standard, and both change the emitted hash: V8's cos differs from glibc's by 1 ulp on ~3.5% of the arguments this DCT uses, which alters ~7% of hashes. Since a thumbhash is sent to other people, that would make it a weak fingerprint of the sending platform. Exact integer argument reduction also happens to be 27x more accurate than the reference formulation, which hands libm a thrice-rounded angle. Verified identical across 7 GCC and 2 Clang configurations spanning -O0..-O3, -march=native, -mfma and -ffp-contract both fast and off.
The basis tables are hoisted out of the innermost loops; the reference rebuilds them per (cx, cy) when encoding and per pixel when decoding. Accumulation order is untouched, so this is bit-identical, and about 3x faster each way: encode 0.11ms at 32x32, decode 0.085ms at 32x24.
Decoding is resolution-independent, so there is no upscale-and-blur step and no image library needed on the receive path. Keep the decode small and let the UI scale it: at 32px, upscaling 10x with any linear filter differs from a full-size decode by RMSE 0.87, which is imperceptible.
The hash therefore differs from upstream encoders by at most 1 in an individual 4-bit AC coefficient. Decoders remain fully interoperable; nothing requires two encoders to agree. THUMBHASH_REFERENCE_COS switches the basis back to the upstream std::cos formulation, which is what the algorithm is defined as and what its published vectors were produced with; it is there to keep that legible and measurable, and must not be defined in production.
Two tests guard the parts that would otherwise fail silently. The pinned hash vectors catch a build that reintroduces a platform dependency, rather than letting it leak. The det_trig case checks the hand-written Taylor coefficients against libm across the DCT's whole argument set, because a mistranscribed factorial still yields smooth, plausible output -- just with the wrong values in every hash.
The API is shaped so the safe path is the obvious one. decode() takes explicit dimensions, because a hash carries no usable record of the source's shape: what it stores is the DCT component counts, which a decoder needs in order to know how many luminance AC coefficients precede the chroma terms. That parameter tracks the shape loosely, so upstream exposes it as an aspect ratio, but one side is always pinned at the per-channel maximum and the complete set of results is 7/n for n in 1..7 and reciprocals (13 values), or 5/n for n in 1..5 with alpha. It also saturates, so a 1000x100 banner reads as 7.0 rather than 10.0 and a 1x100 sliver is wrong by a factor of fourteen. It is therefore named component_aspect_ratio(), and the overload that guesses an output shape from it is decode_unsized(); both are documented as diagnostics for a hash that arrives with no metadata, not as part of a display path.
valid() and expected_size() are for a carrier storing peer-supplied values. A hash's length is not merely bounded but fully determined by its own header -- only six lengths are reachable at all -- so a receive path can check exact structural validity for the cost of a few bit extractions, which is strictly stronger than a length cap that any blob of the right size would pass.
Note for the libvips work: encode() takes RGBA at <=100x100, so the caller does the downscaling, and the scaler is now the weakest link for reproducibility -- two clients that downscale the same photo differently produce different hashes.